Approximating Mutual Information by Maximum Likelihood Density Ratio Estimation
نویسندگان
چکیده
Mutual information is useful in various data processing tasks such as feature selection or independent component analysis. In this paper, we propose a new method of approximating mutual information based on maximum likelihood estimation of a density ratio function. Our method, called Maximum Likelihood Mutual Information (MLMI), has several attractive properties, e.g., density estimation is not involved, it is a single-shot procedure, the global optimal solution can be efficiently computed, and cross-validation is available for model selection. Numerical experiments show that MLMI compares favorably with existing methods.
منابع مشابه
The Development of Maximum Likelihood Estimation Approaches for Adaptive Estimation of Free Speed and Critical Density in Vehicle Freeways
The performance of many traffic control strategies depends on how much the traffic flow models have been accurately calibrated. One of the most applicable traffic flow model in traffic control and management is LWR or METANET model. Practically, key parameters in LWR model, including free flow speed and critical density, are parameterized using flow and speed measurements gathered by inductive ...
متن کاملModified Maximum Likelihood Estimation in First-Order Autoregressive Moving Average Models with some Non-Normal Residuals
When modeling time series data using autoregressive-moving average processes, it is a common practice to presume that the residuals are normally distributed. However, sometimes we encounter non-normal residuals and asymmetry of data marginal distribution. Despite widespread use of pure autoregressive processes for modeling non-normal time series, the autoregressive-moving average models have le...
متن کاملEstimation and Classification by Sigmoids based on Mutual Information
An estimate of the probability density function of a random vector is obtained by maximizing the mutual information between the input and the output of a feedforward network of sigmoidal units with respect to the input weights. Classification problems can be solved by selecting the class associated with the maximal estimated density. Newton's method, applied to an estimated density, yields a re...
متن کاملThe Development of Maximum Likelihood Estimation Approaches for Adaptive Estimation of Free Speed and Critical Density in Vehicle Freeways
The performance of many traffic control strategies depends on how much the traffic flow models are accurately calibrated. One of the most applicable traffic flow model in traffic control and management is LWR or METANET model. Practically, key parameters in LWR model, including free flow speed and critical density, are parameterized using flow and speed measurements gathered by inductive loop d...
متن کاملPREPRINT: PLEASE DO NOT DISTRIBUTE OR CITE Maximum Likelihood Wavelet Density Estimation with Applications to Image and Shape Matching
Density estimation for observational data plays an integral role in a broad spectrum of applications, e.g. statistical data analysis and information-theoretic image registration. Of late, wavelet based density estimators have gained in popularity due to their ability to approximate a large class of functions; adapting well to difficult situations such as when densities exhibit abrupt changes. T...
متن کامل